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Background: Many of the agent-based products available to education today have been largely 
confined to games and instructional media that are often produced commercially by non-instructional 
professionals. Most of them lack customizability and are difficult to integrate into individual subject 
curriculum. With more agent authoring tools available and the proliferation of client-end computing, 
classroom teachers can now author their own agents pedagogically to produce learning environments that meet 
their predetermined objectives. 

Aims: To experiment the use of agent technology with a class of pre-service teachers (aka trainee 
teachers). It hopes to achieve two objectives: (1) to understand how well the agent technology is accepted by 
the trainee teachers, and (2) to reveal any difficulty in designing the agent-based learning environment. 

Sample: 16 pre-service teachers took part in this study. All of them were trainees from an educational 
technology module that had a main component in designing learning environment. They were second year 
students doing a Diploma-in-Education course in their own field of specialization. 

Method: Pre-service teachers were to design agent-based learning environments on their own. A 
questionnaire was used to obtain summative assessment from the participants at the end of the course. The 
score from the questionnaire was used to gauge the trainee teachers’ acceptance for their learning experience. 
Trainee teachers’ weekly self-reporting Weblog was used as a formative assessment on their learning 
processes and difficulties. 

Results: This study found that agent technology was well received by the trainee teachers albeit 
concerns for support by schools and the need to learn programming skills. Generally, trainee teachers were 
enthused by the use of scenes to support their lessons. They regarded their learning experience as positive and 
meaningful. They also believed that such technology would be helpful in promoting interest in their classroom 
teaching. 

Conclusion/Recommendations: Based on the results, agent-based technology is recognized as a 
useful tool to support classroom learning. This study recommends two measures: (1) to build a repertoire of 
customizable scene templates focusing on very small and specific content areas, (2) to organize optional basic 
agent scripting training program prior to the commencement of the course for those who are weak at 
programming skills. 

Keywords: Pedagogical agent, agent-based learning, learning environment. 
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1. Introduction 

Designing learning environments 
involving pedagogical agents is new to 
the teachers in Singapore. Despite strong 
evidence of positive results reported by 
many studies conducted in the West (e.g. 
Baylor, 2002; Buisine, Abrilian, & Martin, 
2004; Clarebout, Elen, Johnson, & Shaw, 
2002; Johnson, Rickel, & Lester, 2000; 
White, Frederiksen, Frederiksen, Eslinger, 
Loper, & Collins, 2002), the adoption of 
agents for designing learning environment 
for classroom use is slow and limited. It is 
in this light that this study is initiated. It 
involved a class of pre-service teachers 
taking a course in Diploma in Education 
at the National Institute of Education. The 
main purpose of the study is to find out 
how instructional design may incorporate 
agent technology and how well trainee 
teachers perceive their learning using this 
new technology. 

There are some initial concerns 
about teaching trainee teachers the use of 
agent technology. First, the demand on 
technical competency. Trainee teachers 
need to have two basic skills - the agent 
scripting skill and basic HTML coding 
skill in order to animate the agent. This is 
a challenging task for those who have 
little or no prior programming experience. 
Second, the use of scenes to contextualize 
the learning process. This is rather new to 
most of the trainee teachers and can be 
demanding for some. Scenes, like cases, 
scenarios and problems, have the potential 
to provide realistic settings for 
meaningful learning by situating learners 
in authentic activities (Duffy & Jonassen, 
1992). Last, the task of integration, which 
requires the trainee teachers to put 
together the various components 


(animations, speech, dialogue and scenes) 
with sound pedagogy to produce a 
meaningful agent-based learning 
environment. All these requirements are 
fundamental to the proper integration of 
ICT in teaching and they are the key 
issues for this paper. For research purpose, 
these issues are condensed into two 
questions which will be answered in the 
respective sections that follow. The two 
questions are: 

1.1 What is the acceptance level of 
agent-based instruction reported by the 
trainee teachers? 

1.2 What are the difficulties in 
designing agent-based learning 
environments? 

2. Designing an Agent-based 
Learning Environment 

2.1 The Model for the Design 

Traditionally, most instructional 
design courses use ADDIE (see Figure 
2.1) as a model to introduce instructional 
design processes. This model was popular 
in the eighties because of its 
straightforwardness and the strong 
support it received from instructivistic 
practitioners. However, with the advent of 
constructivism, it now faces criticism for 
the lack of objectivity and the inability to 
produce meaningful learning process 
(Mendonca, 2003). ADDIE model, 
although appear to lose ground from the 
constructivist camp; is still a useful 
instructional model to many ‘moderate’ 
constructivists, especially to the designers 
of online courses. This being so because 
online learning is still very much 
individualistic and deals primarily with 
elementary content knowledge which is 
easier to learn in an instructivist’s 
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environment. This paper, thus proposes 
adopting the ADDIE model but with 

Figure 2.1: ADDIE Model 



Two modifications were done to 
the original ADDIE model. First, the 
components “Implement” and “Evaluate” 
were removed from the model because 
trainee teachers did not have the 
opportunity to implement their designs in 
a real school setting. Second, an 
additional element called “EPTT” (see 
Table 2.1) was added to complement the 


modifications to accommodate the 
constructivists’ requirements. 

component “Analyse”. “Analyse” refers 
to analysing the learners’ attitude, skills 
and knowledge prior to taking the course 
and are collectively called the “ASK” of 
learners. It was felt that the requirement to 
know the learners’ pre-learning conditions 
was insufficient because it disregards the 
designer’s epistemological belief which 
has a strong bearing on the designer’s 
expectations and hence his/her approaches. 
Designers who understand their 
epistemological stance tend to be more 
aware of the limits of their pedagogy and 
are more likely to produce meaningful 
and logical instructional products. With 
the introduction of EPTT, a more 
balanced design process between learners’ 
characteristics and the designer’s 
expectations may be possible. Hence, this 
paper suggests that “Analyse” should 
involve the analysis of both learners and 
designer. 


Table 2.1: The EPTT 


Domain to consider 

What to apply? 

Epistemological 

Constructivism, Activity Theory, 

Perspective 

1 1 

Experiential Learning etc. 

Pedagogical 

Case-based, Problem-based, Exploratory, 

Paradigm 

Research-based etc. 

1 ▼ 

Theoretical 

Multimedia Learning Theory, Information 

Basis 

1 1 

Processing Theory etc. 

▼ ▼ 

Tools 

1. Scene for contextualization 

2. Microsoft Agents for scaffolding 


EPTT in Table 2.1 stands for 
'Epistemological Perspective”, 

Pedagogical Paradigm”, “Theoretical 


Basis” and “Tools”. EPTT requires the 
designer to first examine his/her own 
belief of learning (Epistemological 
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Perspective) before the design. The 
designer then looks for proper 
pedagogical approaches to support his/her 
epistemology. For example, a 
constructivist designer would probably 
choose cases or problems for his/her 
instructions. (Jonassen, 2000; Jonassen, 
Howland, Moore, & Marra, 2003; Smith 
& Ragan, 1999). Next, the designer will 
have to identify a relevant theory to guide 
the design. Finally, the designer needs to 
select instructional tools to facilitate the 
learning process. A point to note is that 
because agents and scenes are part of the 
intervention in this empirical study, all 
trainee teachers are supposed to include 
them as facilitating tools in their designs. 
Facilitation is an essential process for 
effective learning (Davis, & Davis, 1998; 
Schunk, 2000). Agents and scenes will be 
discussed in detail in the following 
sections. 

2.2 Pedagogical Agents and Microsoft 
Agents 

Since the existence of agent in 
education, the term “agent” has received a 
number of different semantics depending 
on the role it plays. Some called it 
instructional agent (Clarebout et al., 2002); 
others called it pedagogical agent (Baylor, 
2002; Clarebout et al., 2002; Craig, 
Gholson, & Driscoll, 2002; Johnson et al., 
2000; Kristen, Person, Adcock, van Eck, 
Jackson, & Marineau, 2002; Smith, 
Afflect, Lees, & Branki, 1999) or 
intelligent agent (Goecks & Shavlik, 
2000). Shaw, Johnson, & Ganeshan (1999, 
p. 1) defined pedagogical agents as 
“animated characters designed to operate 
in an educational setting to support or 
facilitate learning”. This matches the 
scaffolding role the agent needs to play in 
this study. Thus, pedagogical agent will 
be used in the subsequent discussions in 
this paper. 

The effect of pedagogical agents 
on learning has been equivocal because of 
the complex nature of involvement it 
offers in the learning process. Some used 


agents as mediating tools while others use 
them to increase productivity. But among 
all, one strong attribute stood out against 
the rest is its ability to influence 
positively a learner’s perception of 
learning. Many believe that agents are 
useful not because of their ability to 
animate but their capacity for speech and 
the ability to mimic lifelike behaviours - a 
characteristic described as persona. A 
persona is a personified agent that co- 
exists with the user in a learning 
environment (Mulken et at., 1998). By 
personifying the agent, a user tends to 
believe that the agent is “real” and helpful, 
and as a result, enjoys a more positive 
learning experience (e.g. Baylor & Ebbers, 
2003; Lester, Converse, Kahlet, Barlow, 
Stone, & Bhogal, 1997; Mulken, Andre, 
& Muller, 1998). Supporting findings for 
persona effect includes Johnson’s et al. 
(2000) study on agent-based interactive 
learning environment. They found that 
agents with humanlike behaviour such as 
gaze and deictic gesture were more 
effective in motivating learning. A 
plausible explanation to this is that 
behaviour aims at fulfilling certain 
instructional purposes tends to lighten the 
cognitive load of the learners. The effect 
is equivalent to having a tutor or 
facilitator in a learning environment. 
Learners with low cognitive load 
normally suffer less fear for failure and 
are therefore keener to learn - a situation 
described by an OECD (2002) report as 
“high challenge” with “low threat”, an 
environment regarded as ideal for 
learning. 

In another study, Lester, Stone, & 
Stelling (1999) investigated problem- 
based learning facilitated by lifelike 
pedagogical agents. They found that 
learners learn better in solving complex 
problems when the learning path was 
assisted by the agent’s interactive 
feedback and timely advice accompanied 
by well-tuned human behaviour. They 
cautioned that an agent’s behaviour must 
be linked to its role in facilitation within 
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the context, in particular, to engage 
learners in a variety of explanatory, 
advisory and believability-enhancing 
behaviours. In other words, the finding 
pointed out the importance of context and 
the need to have well thought-out 
instructional strategies. 

In all, the above findings 
demonstrate common ideas of what 
constitutes to a conducive agent-based 
learning environment, these are: right 
choice of context, proper coordination of 
agent’s speech, movements and gestures, 
and appropriate pedagogical approaches. 

There is a need to include one 
more consideration for this study, that is, 



the choice of agent. Because the bulk of 
the trainee teachers are new to agent 
programming, the agents must be simple 
and easy to animate. There is hardly any 
such agent until 1992 when Microsoft 
Corporation officially released its 
proprietary agent technology to the public 
(Wissick, 2002). Since then, Microsoft 
agents have been widely incorporated in 
business such as web portals and 
presentations. But its use in education is 
still limited. There are altogether four 
agents available for public use; they are 
Peedy, Merlin, Genie and Robby (see 
Figure 2.2). 



Figure 2.2: Microsoft Agents (from left: Peedy, Merlin, Genie and Robby) 


Microsoft agents contain features 
that are very suitable for producing agent- 
supported learning environment at 
classroom level. There are several reasons. 
First, these agents are customizable and 
re-creatable. Anyone who wants to create 
a new agent character (other than the 
given four) can use the development kit 
downloadable from Microsoft 
Corporation. Second, the agent is fully 3- 
D animated and has preprogrammed 
lifelike behaviour. Lifelike behaviour is 
important to induce “persona” effect. Also, 
the animation is quite straightforward, 
using a near English-like scripting 
language. Third, the agent is able to 
deliver synthesized speech and respond to 
spoken commands, a very friendly tool 
for those who need special attention. 
Fourth, it can be programmed to interact 
with the environment to enhance human- 


computer interface through the use of 
specialized software. Last, which is also 
the most important one; it is license-free 
and can be installed in any window-based 
computers. This allows the trainee 
teachers to practise the agent 
programming skill at their own time at 
their home. This is considered crucial for 
new learners. Since Microsoft agents 
possess many of the important ingredients 
that match the requirements for designing 
an agent-based learning environment, 
they are adopted and used in this study. 

2.3 Use of Scenes 

Scenes are commonly used in 
plays, dramas and movies. Besley, 
Bhangal, & Farr (2002) called a scene an 
independent piece of a whole movie that 
helped to advance the storyline. When a 
movie gets too complex or lengthy, 
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scenes play the role of segmenting the 
storyline into more manageable 
components and yet maintaining the flow 
and continuity (Gooch, 1988). In the like 
manner, when scenes are used in 
instructions, they can help focus the 
essentials, provide in-depth study on the 
parts rather than the whole for 
complicated issues. 

To produce scenes, the following 
elements suggested by Selvester’s (2004) 
are required: 

1. Dialogue 

2. Actions and gestures 

3. Characters’ thoughts 

4. Expositions 

5. Description of setting 

6. Comments or observations by the 
author or narrator 

7. Transitions from the previous 
scene into the next scene 

8. Contribution to the plot 

9. Successful or failed outcomes 
Many of the above elements can 

be fulfilled by using Microsoft agents. For 
example, an agent can play a character in 
a scene, converse with another agent with 
gestures and movements to induce a 
conflict or lay the background of an issue 
which can then form the context for 
learning. Scene-based learning is 
analogous to constructivist learning. 
Scene-based learning uses scenes as a 
catalyst to scaffold whereas constructivist 
learning uses tools such as case study and 
problems to support learning (Davis, & 
Davis, 1998; Jonassen, 1998; Jonassen, 
2000; Pieters, 1995; Schunk, 2000). Both 
scenes and constructivist environments 
emphasize authenticity but scenes are able 
to provide a more realistic situation by its 
characters’ interplay and dialoging. More 
importantly, scenes can be fabricated to 
achieve a certain goal without 
compromising authenticity. 

2.4 Producing the Learning 
Environment 

Previous discussions outlined the 


tools and components necessary for 
designing and constructing the agent- 
based learning environment. In 
consolidation, what the trainee teachers 
need to do in the course is to first 
understand the design model given in 
Figure 2.1 and Table 2.1. They then use 
the model to identify a very specific idea 
in the content area which they find it 
difficult to teach, e.g. human adaptation to 
environmental changes, measures to curb 
dolphin’s extinction etc. Two things they 
have to take into consideration when 
making their choices: (1) choose only 
ideas that can be acted out through the use 
of scenes and (2) choose ideas that can be 
learnt through a problem or an issue. 
Once the idea to learn is identified, the 
trainee teachers will need to work out a 
situation containing the problem or issue. 
The problem or issue will be segmented 
into smaller and manageable episodes 
called scenes. The scenes are then 
introduced to the learners with the help of 
agents. Agents will be programmed to 
play the role of a facilitator in the scenes, 
highlighting the crux of the problems 
through their voices and animations. The 
learning process will culminate in 
activities such as group discussion or 
presentation. 

3. Methodology 

3.1 The Participants 

A group of 16 pre-service teachers 
took part in this study. The pre-service 
teachers were trainees from an 
educational technology module that had a 
main component in designing learning 
environment using agent technology. All 
trainees were second year students doing 
a Diploma-In-Education course in their 
own field of specializations. The 
breakdown of the trainees’ background in 
terms of Content Specialization 
(abbreviated as CS) and gender is given in 
Table 3.1. 
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Table 3.1 : Breakdown of Trainees’ background 


Content 

No. of Male 

No. of Female 

Sub-total 

Specialization 

Participants 

Participants 

General 

2 

6 

8 

Art 

2 

0 

2 

Home 

Economics 

0 

1 

1 

Chinese 

0 

2 

2 

Malay 

0 

3 

3 

Sub-total 

4 

12 

Total = 16 


3.2 Implementing the Course 

The course consisted of ten in- 
class and two off-class sessions. Eight out 
of the ten in-class sessions were devoted 
to designing agent-based learning 
environment. The rest (including off-class 
sessions) were lessons focusing on giving 
general concepts on instructional 
strategies, agent’s roles and supports in 
learning environments. Trainee teachers 
spent all the eight sessions face-to-face 
including learning a new tool called 
MASH (a specialized tool used for 
scripting Microsoft agents), understanding 
the concept of scene, practising the 
scripting skills and assembling the parts 
into deliverables. 

To obtain regular feedback from 
the trainee teachers on their learning 
experience, the course required the trainee 
teachers to write weekly reflection in the 
form of Weblog (aka Blog) on what they 
had learnt, what learning difficulties they 
had encountered and what their 
suggestions were for improvement. 
Reflections were uploaded to Blogspot, a 
public domain web-based platform for 
hosting self-reported journals. Trainee 
teachers were told that blogging was 
graded and formed part of the total 
assessment for the course. 

At the end of the course, trainee 
teachers were to produce a web-based 
integrated agent-based project in a subject 
discipline of their choice. The project 
carried 50% of the final grade. 


3.3 The Research Design 

This study adopted an evaluative 
design to elicit information pertaining to 
the trainees’ reported learning experience. 
A questionnaire was used as an 
instrument to obtain summative 
assessment from the participants at the 
end of the course (Bell & Opie, 2002). 
This provides a means to answer research 
question 1. Trainee teacher’s weekly self- 
reporting Weblog was used as a formative 
assessment to obtain trainee teachers’ 
learning experiences and their general 
comments on learning. This serves to 
answer research question 2. Finally, the 
overall score from the questionnaire was 
used to measure the trainee teachers’ self- 
reported learning profile (aka perceived 
learning profile). The profile is used as a 
means to gauge the trainee teachers’ 
acceptance for their learning experience. 

3.4 The Questionnaire 

A questionnaire consisting of 36 
self-constructed items was used. It has 
four subscales: (1) usefulness of MASH, 
(2) proficiency in scripting the agents, (3) 
perception of the scene concept and (4) 
perception of scene -based activities. All 
subscales contained 5-point Likert-type 
items and one open-ended question. Each 
item consists of five options: Strongly 
Agree, Agree, Not Sure, Disagree and 
Strongly Disagree. Table 3.2 gives the 
breakdown of the questionnaire structure. 
The open-ended question allowed 
feedback on areas that were not covered 
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by the Likert-type items. The last session of the course. There was no 
questionnaire was administered during the time limit for taking the questionnaire. 


Table 3.2: Breakdown of Questionnaire Structure 



Subscale 

Usefulness of 

MASH 

(UM) 

Proficiency in 
scripting the 
agents 
(PSA) 

Perception of 
the scene 

concept 
(PSC) 

Perception of 
scene -based 
activities 
(PSBA) 

No. of Likert- 
type item 

7 

10 

6 

9 

No. of Open- 
ended question 

1 

1 

1 

1 

Sub-total 

8 

11 

7 

10 


3.5 Trainee Teachers’ Perceived 
Learning Profile (LP) 

Positive perception has long been 
known to be related to meaningful 
learning (Eggen & Kauchak, 1999) and 
meaningful learning is likely to produce 
desirable educational outcome. It is 
therefore important to know how trainee 
teachers perceive their learning 
experience. The total score from the 
questionnaire is used as a surrogate 
measure for the trainees’ self-reported 
learning experience. Understandably, the 
score reflects the trainees’ own belief of 
learning which can be a result of 
subjectivity. The score from the 
questionnaire was called perceived 
learning profile, abbreviated as “LP”. 

4. Data Analysis and 
Discussions 

4. 1 The Questionnaire 

The questionnaire is an important 
instrument in this study. It must meet 
acceptable criteria if the information it 
collects is to be useful. Below are its 
reliability and validity. 

4.1.1 Reliability 

Six items were removed from the 


original questionnaire either because of 
non-contribution to variance or low item- 
total correlation (r<.3) (Nunnally & 
Bernstein, 1994). The reminding 30 
Likert-type items and four open-ended 
questions then formed the revised 
questionnaire to be used in the study (see 
Appendix A). The revised questionnaire 
had a Cronbach’s Alpha of .891 which 
fell well within the acceptable range of .6 
to .9 (Aron & Aron, 2003). 

4.1.2 Validity 

The divergent validity of the 
questionnaire was verified by examining 
the inter-correlations between the 
subscales (Nunnally & Berstein, 1994). 
The result is tabulated in Table 4.1. It can 
be seen that all subscales correlated 
significantly with LP implying that all 
subscales were able to account 
significantly the variance of the perceived 
learning. All subscales also had 
insignificant correlations with one another 
except for the pairs PSC-UM (r = .631) 
and PSC-PSA (r = .629). This could be 
due to the fact that scenes are perceived to 
be closely tied to its application which is 
also strongly linked to the tools and skills 
that mediate the application. 
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Table 4.1: Inter-correlations of Subscales 



UM 

PSA 

PSC 

PSBA 

LP 

UM 

PSA 

PSC 

PSBA 

LP 

1 

.378 

.631 (") 

.159 

. 726 (") 

.378 

1 

. 629 (") 
.329 
.751 (") 

.631 (") 
. 629 (") 
1 

.476 

. 900 (") 

.159 

.329 

.476 

1 

. 657 (") 

. 726 (") 
.751 (") 
. 900 (") 
. 657 (") 
1 


''Correlation is significant at the .01 level (2-tailed). 


4.2 Acceptance Level of Agent-based 
Instruction Reported by the Trainee 
Teachers 

Acceptance level was measured by 


the means of the subscales and the total 
scores of the questionnaire. Table 4.2 
contains the means and relevant statistics 
for the overall scale and subscales. 


Table 4.2: Descriptive Statistics for Overall Scale and Subscales 



Minimum 

score 

Maximum 

score 

Mean 

score 

Percent Mean 

Std. Deviation 

UM 

12 

28 

25.38 

72.5 

3.76 

PSA 

18 

29 

22.69 

75.6 

2.96 

PSC 

17 

30 

24.75 

82.5 

3.42 

PSBA 

22 

35 

28.50 

81.4 

3.48 

LP 

76 

122 

101.31 

72.4 

9.74 


Trainee teachers seemed to accept 
the use of scene concept (PSC) in agent- 
based environments more than the others 
(see Percent Mean). This is followed by 
the use of scene -based activities (PSBA) 
and the proficiency in scripting the agents 
(PSA). The high PSC score (82.5%) 
which measures the understanding of 
scene usage and scene effectiveness, is a 
strong indication that the scene concept is 
well explained to and well accepted by 
the trainee teachers. This tends to imply 
that the revised ADDIE model is useful 
and effective in conveying the needs to 
analyse both the learners and designer. 
The introduction of EPTT could have 
helped in this regard by lightening the 
cognitive load of trainee teachers. By 
following an organized structure of 
thought which the EPTT aims to provide, 
trainee teachers are more aware of the 
nature of the process and are more 
confident in their doing. This 
phenomenon is consistent with the ideal 
learning condition characterized by a 
“high challenging but low threatening” 
environment described by the OEDC 
(2002) report. Of course, the result could 


be better verified by comparing the 
trainee teachers’ perceived learning with 
their actual performance but this would 
face one major constraint, that is, the 
performance would not be linked to 
applicability of product because the 
deliverables cannot be tested in a real 
classroom situation. As in all research 
studies, design effectiveness needs to be 
tested for practicality or the value of 
assessment would not hold water if 
performance is to be measured solely by a 
set of classroom rubrics. With this 
constraint, this study limits the 
measurement to only perceived learning. 

The high range of subscale scores 
also help to allay the initial concerns 
highlighted in the Introduction part 
(section 1) of this paper. The statistics did 
not reveal any adverse tendency to 
learning new concepts and technical skills. 
Indeed, it is quite surprising to learn that 
technical ability and the usefulness of the 
scripting tool were rated unexpectedly 
high in a course like this which had so 
many novelties to learn. It is also very 
encouraging to see that the reported 
perceived learning (LP) stood at a 72.4%. 
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To a certain extent, the result could be 
taken as a form of acknowledgement to a 
satisfactory introduction of agent-based 
technology in instructional design. 

4.3 Difficulties in Designing Agent- 
based Learning Environments 


Analysis of the trainee teachers’ 
weekly reflections and the open-ended 
questions from the questionnaire provided 
the source to know their learning 
problems. The main concerns raised by 
the trainee teachers were tabulated in 
Table 4.3. 


Table 4.3: Main Concerns Regarding the Agent-based Learning 

Trainee Teachers’ Concerns Pertaining to Learning the Course 

Concerns related to UM 

1. Access to the MASH software is only confined to NIE’s laboratories. Using the trial version software 
allows only a month of usage. 

2. MASH does not support speech in Chinese. It does not favour trainee teachers specializing in Chinese 
teaching. 

Concerns related to PSA 

1 . It takes great effort to script the agents as the program does not tolerate any slightest mistake. This makes 
the scripting very time-consuming. 

2. I am weak in programming so I wish to have longer hands-on practice time in class. 

3. Hope to have readily made agent templates that can be adopted and adapted for a rapid prototype lesson. 

Concerns related to PSC 

1. Scene is good in attracting students’ interest but to have good scenarios or events to move the scenes needs 
good content knowledge and some creativity. 

2. Need a good deal of guidance and consultation from the tutor. Not all topics are suitable for use in scene- 

based environment. 

Concerns related to PSBA 

1. Not enough time to complete the project and design the activities. Especially trainees with Chinese 
specialization need extra effort to translated the scene events. 

2. Creating agent-based learning environment requires manpower and time. May need a team rather than an 
individual to do it. 

General Concerns 

1. Not sure whether the delivery of the agent -based instructions would work in a school setting given the 
number of engine software (agent and speech) to install and the technical support available. 

2. Prefer more intelligent agents like Peedy (one that has the most animations in MS agents) and more agent 
type to choose from. 


The feedback pointed out two key 
areas of concern, the practicability and 
availability. A few asked whether such a 
learning environment is practical in 
school settings given the time and support 
needed. This worry is not without grounds. 
Normally, any new implementation in 
schools would not involve only the 
adaptation for the change but also a 
change in the mindset of the affected 
bodies. How well a change is perceived as 
acceptable will always depend upon the 
extent of need to make the change. Agent- 
based instruction is entirely new to the 
school and is expected to receive mixed 
reactions. Other trainee teachers queried 
about the extent of provision in terms of 
software and guidance. Software issue is 


an institutional issue and is not easy to 
tackle. Having more technical guidance is 
probable once agent-based instruction 
becomes integrated in core technology 
modules. In addition, a pre-course 
tutoring system can be implemented to 
support those with weak programming 
background. Despite these issues, the 
questionnaire indeed received a good 
number of positive feedback such as 
scenes help to engage metacognition, 
agents are able to entice students’ interest 
and that the revised ADDIE model helped 
greatly in shaping their ideas of 
instructional design. The favourable 
comments also appear to agree with the 
high mean scores found in the subscales 
of the questionnaire. 
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5. Conclusion and 
Recommendations 

This study attempts to address two 
research questions pertaining to pre- 
service teachers’ acceptance of agent- 
based instruction, i.e. how well the trainee 
teachers perceive their learning and what 
their related learning difficulties are. 
Questionnaire and weekly reflections 
were the main mechanism used to collect 
data. Following are the findings obtained: 

Generally, the new approach of 
instruction using the agent technology and 
scene concept was well received by the 
trainee teachers. No major issues were 
reported except for two: (1) the 

practicability and availability, and (2) the 
technical skills. Trainee teachers were 
eager to use the new technology but were 
skeptical of the support and time available 
to them. From a pedagogical point of 
view, the concern may be addressed by 
building a repertoire of customizable 
scene templates focusing on very small 
and specific content areas. This helps to 
ease the need to construct the instruction 
from scratch. Over time, hopefully the 
templates can grow into a resource 
repository and attract a community of 
users. 

A few trainee teachers felt that the 
scripting and programming skills were too 
demanding. Since the course admission 
policy does not allow screening the 
trainees for technical skills, this study 
recommends organizing optional basic 
scripting training program prior to the 
course for those who are technically less 
inclined. 

The introduction of EPTT to the 
original ADDIE model seems to help 
trainee teachers understand the design 
process and scene concept well. Having 
the designer to consider his/her 
epistemological perspective of learning 
alongside with the design process appears 
to make the learning task easier. This is so 
because designing an instructional process 
using a very new technology entails a lot 


of guts and initiatives. The EPTT in a way 
provides a structure for a beginner to start 
with. This enhances their self-confidence 
which may translate into higher 
acceptance of their learning experience. 

Overall, the positive perception of 
learning involving agent-based 
instructions is a good indication to run 
similar courses in the future. However, 
care must be taken to emphasize the 
pedagogical purpose and instructional 
value of the design and possibly, to 
provide scripting and technical skill 
training prior to the course proper for 
those technically less inclined trainees. 

References 

Aron, A., & Aron, E. N. (2003). Statistics for 
psychology (3 ld ed.). NJ: Prentice Hall. 

Baylor, A. L. (2002). Expanding preservice 
teachers’ metacognitive awareness of 
instructional planning through pedagogical 
agents. Educational Technology , Research 
and Development, 50(2), 5-22. 

Baylor, A. L., & Ebbers, S.J. (2003, June). The 
pedagogical agent split-persona effect: 
when two agents are better than one. Paper 
presented at the ED-MEDIA 2003, World 
Conference on Educational Multimedia, 
Hypermedia & Telecommucations, 
Honolulu, HI. 

Bell, J., & Opie, C. (2002). Learning from 
research: getting more from your data. 
Buckingham: Open University Press. 

Besley, K„ Bhangal, S„ & Farr, A. (2002). 
Foundation Flash MX. Birmingham: 
Friends of ED. 

Buisine, S., Abrilian, S., & Martin, J. C. (2004). 
Evaluation of multimodal behaviour of 
agents. Retrieved November 29, 2004 
from http://www.niceproject.com/publications 
/buisine-et-al_03 1 1 .pdf 

Clarebout, G., Elen, J., Johnson, W. L., & Shaw, E. 
(2002). Animated pedagogical agents: an 
opportunity to be grasped? Journal of 
Educational Multimedia and Hypermedia, 
11(3), 267-286. 

Craig, S. D., Gholson, B., & Driscoll, D. M. 
(2002). Animated pedagogical agents in 
multimedia educational environments: 
effects of agent properties, picture features 
and redundancy. Journal of Educational 
Psychology, 94(2), 428-434. 

Davis, J. R. & Davis, A. B. (1998). Effective 
training strategies - a compressive guide 
to maximizing learning in organization (pp. 


49 



Huay Lit WOO, Qiyun WANG 


382-384). San Francisco: Berrett-Koehler 
Publisher. 

Duffy, T. M. & Jonassen, D. H. (1992). 
Constructivism: New implications for 

instructional technology. In T. M. Duffy, 
& D. H. Jonassen (Eds.). Constructivism 
and the technology of instruction: a 
conversation (pp. 1-16). NJ: Lawrence 
Erlbaum Associates. 

Eck, R. V. & Adcock, A. (2004). Attitude toward 
non-intelligent agent scale. Retrieved 
November 29, 2004 from http://idLmenphLs.edu/ 
~rvaneck/AERA2003 .pdf 

Eggen, P., & Kauchak, D. (1999). Educational 
Psychology (4 th ed.). NJ: Prentice-Hall. 

Elen, J. (1995). Blocks on the road to instructional 
design prescriptions. Leuven, Belgium: 
Leuven University Press. 

Goecks, J., & Shavlik, J. (2000, January). 
Learning users’ interests by unobtrusively 
observing their normal behavior. Paper 
presented at the International Conference 
on Intelligent User Interfaces, New 
Orleans, LA. 

Gooch, S. (1988). Writing a play. London: A & C 
Black. 

Johnson, W. L., Rickel, J. W., & Lester, J. C. 
(2000). Animated pedagogical agents: 
face-to-face interaction in interactive 
learning environments. International 
Journal of Artificial Intelligent in 
Education, 11, 47-78. 

Jonassen, D. H. (1998). Designing constructivist 
learning environments. In C. M. Reigeluth 
(Ed.), Instructional theories and models 
(2 nd ed.). NJ: Lawrence. 

Jonassen, D. H. (2000). Computers as mindtools 
for schools - Engaging critical thinking. 
NJ: Prentice-Hall. 

Jonassen, D. H., Howland, J., Moore, J., & Marra, 
R. M. (2003). Learning to solve problems 
with technology. NJ: Pearson Education. 

Lester, J. C., Stone, B. A., & Stelling, G. D. 
(1999). Lifelike pedagogical agents for 
mixed-initiative problem solving in 
constructive learning environments. User 
Modeling and User-adapted Interaction, 9, 
1-44. 

Mayer, R. E. (2001). Multimedia learning. 
Cambridge, England: Cambridge 

University Press. 

Mendonca, A.L.R. (2003, April). The practice of 
instructional design: The process and its 
application. Proceeding of PGL DB 
Research Conference, Brazil, 20-32. 

Moreno, K. N„ Person, N. K„ Adcock, A. B., Van 
Eck, R. N., Jackson, G. T., & Marineau, J. 
C. (2002, November). Etiquette and 
efficacy in animated pedagogical agents: 
The role of stereotypes. Paper presented at 


the 2002 AAAI Fall Symposium on 
Etiquette for Human Computer Work, 
Cape Cod, MA. 

Mulken, S. V., Andre, E„ & Muller, J. (1998, 
September). The persona effect: how 

substantial is it? Proceeding of HCI on 
People and Computers XIII, UK, 53-66. 

Nunnally, J. C„ & Bernstein, I. H. (1994). 
Psychometric theory (3 ld ed.). NY: 
McGraw-Hill. 

OEDC (2002). Understanding the brain: towards 
a new learning science. France: OEDC 
Publications. 

Pieters, J. M. (1995). Supporting teachers and 
learners to design powerful learning 
environments. In B. Collins, I. Nikolova, 
& K. Martcheva (Eds.). Information 
technologies in teacher education (pp. 40- 
51). Paris: UNESCO. 

Schunk, D. H. (2000). Learning theories - an 
educational perspective (3 rd ed.). NJ: 
Prentice-Hall. 

Selvester, M. N., (2004), Writing fiction: A 
beginner’s guide. Part 7: Scenes and half- 
scenes. Retrieved August 7, 2004, from 
http://teenwriting.about.com/library/weekl 
y/aal 11 102g.htm 

Shaw, E., Johnson, W. L„ & Ganeshan, R. (1999, 
May). Pedagogical agents on the web. 
Paper presented at the third International 
Conference on Autonomous Agents, 
Seattle, WA. 

Smith, P. L., Ragan, T. J. (1999). Instructional 
Design. NY: John Wiley & Sons. 

Smith, T„ Afflect, G„ Lees, B. & Branki, C. (1999, 
December). Implementing a generic 
framework for a web-based pedagogical 
agent. Paper presented at the Annual 
Australassian Society for Computers in 
Learning in Tertiary Education Conference, 
Brisbane, Queesland. 

White, B., Frederiksen, J., Frederiksen, T., 
Eslinger, E„ Loper, S. & Collins, A. (2002, 
October). Inquiry island: affordances of a 
multi-agent environment for scientific 
inquiry and reflective learning. Paper 
presented at the Fifth International 
Conference of the Learning Sciences 
(ICLS). Seattle, WA. 

Wissick, C. (2002). Software and book review 
column: agent-based technology. Journal 
of Special Education Technology, 77(4), 
49-51. 

Woo, H. L. (2003, November). Peers’ Influence 
and Interference on Student-to-Student 
Online Discussion - A Case Study. Paper 
presented at the Educational Research 
Association of Singapore Conference 
(ERAS), Singapore. 


50 



New Horizons in Education, No 52, November 2005 


Appendix A 

Results of Item Analysis for Internal Consistency 
Note: 

1. Q8, 19, 26 and 36 are open-ended questions. They are not included in Table 1 to 4. 

2. Q16 did not produce any variance and is excluded from Table 2. 

3. Items in bold print are to be discarded. 

Table 1: Item-Total Statistics for Subscale UM 



Scale Mean it 
Item Deleted 

Scale 

Variance if 

Item Deleted 

Corrected 

Item-Total 

Correlation 

Squared 

Multiple 

Correlation 

Cronbach's 
Alpha if Item 
Deleted 

Q1 

21.50 

12.400 

.665 

.575 

.860 


21.44 

8.929 

.874 

.837 

.810 


21.81 

11.096 

.626 

.631 

.850 

Q4 

21.56 

12.263 

.409 

.314 

.873 

Q5 

22.94 

11.396 

.375 

.500 

.887 

Q6 

21.50 

9.067 

.801 

.844 

.823 

Q7 

21.50 

9.067 

.906 

.838 

.805 


Table 2: Item-Total Statistics for Subscale PSA 



Scale Mean if 
Item Deleted 

Scale 

Variance if 

Item Deleted 

Corrected 

Item-Total 

Correlation 

Squared 

Multiple 

Correlation 

Cronbach's 
Alpha if Item 
Deleted 

Q9 

30.31 

7.696 

.519 

.423 

.670 

Q10 

30.44 

6.796 

.582 

.504 

.652 

Q11 

31.44 

5.863 

.674 

.592 

.625 

Q12 

30.44 

10.129 

-.047 

.111 

.738 

Q13 

30.44 

9.463 

.168 

.655 

.726 

Q14 

30.75 

7.133 

.465 

.602 

.684 

Q15 

30.31 

9.163 

.589 

.801 

.695 

Q17 

30.31 

8.763 

.442 

.664 

.692 

Q18 

30.56 

9.463 

.198 

.425 

.722 


Table 3: Item-Total Statistics for Subscale PSC 



Scale Mean if 
Item Deleted 

Scale 

Variance if 

Item Deleted 

Corrected 

Item-Total 

Correlation 

Squared 

Multiple 

Correlation 

Cronbach's 
Alpha if Item 
Deleted 

Q20 

20.63 

10.383 

.321 

.800 

.898 

Q21 

20.56 

9.996 

.592 

.840 

.873 

Q22 

20.56 

7.196 

.844 

.782 

.819 

Q23 

20.75 

6.600 

.833 

.855 

.825 

Q24 

20.75 

7.267 

.848 

.887 

.819 

Q25 

20.50 

8.400 

.707 

.798 

.847 


51 

















Huay Lit WOO, Qiyun WANG 


Table 4: Item-Total Statistics for Subscale PSBA 



Scale Mean if 
Item Deleted 

Scale 

Variance if 

Item Deleted 

Corrected 

Item-Total 

Correlation 

Squared 

Multiple 

Correlation 

Cronbach's 
Alpha if Item 
Deleted 

Q27 

33.25 

1 1 .400 

.703 

.832 

.650 

Q28 

33.13 

11.717 

.658 

.751 

.660 

Q29 

33.19 

9.763 

.504 

.970 

.652 

Q30 

33.19 

10.562 

.356 

.980 

.689 

Q31 

33.25 

10.200 

.444 

.906 

.667 

Q32 

32.81 

12.696 

.133 

.917 

.719 

Q33 

33.13 

11.983 

.376 

.924 

.683 

Q34 

33.00 

13.200 

-.023 

.861 

.755 

Q35 

33.56 

9.463 

.664 

.939 

.614 
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